The present invention relates to digital color photography and, more particularly, to a method of adjusting pixel colors to more closely match the expected color of an object depicted in a digital picture.
The human visual system is very sensitive to color shade variations in certain color domains, such as skin colors and sky colors. As a result, when a portion of an image that is supposed to depict skin or sky includes unnatural colors, the color discrepancy is easily detected and causes a degradation in the perceived image quality. It therefore is important to reproduce colors such as skin colors and sky colors with enough fidelity to please the perception of an observer. This task is very demanding when the images concerned are video frames and the required color enhancement, including both detection and correction, must be done in real time. Prior art methods either compromise the accuracy of the detection of unnatural colors or implement complex enhancement algorithms that are computationally too intensive for real time video applications.
Prior art patents and patent applications related to the detection of skin color include:
U.S. Pat. No. 5,585,860 to Takeshima
U.S. Pat. No. 6,072,526 to Hashimoto et al.
U.S. Pat. No. 6,259,426 to Harada et al.
U.S. Pat. No. 6,272,239 to Colla et al.
U.S. Pat. No. 6,476,820 to Harada et al.
U.S. Pat. No. 6,678,407 to Tajima
U.S. Pat. No. 6,690,822 to Chen et al.
US 2003/0142879 to Kim
There is less prior art concerning skin tone enhancement. The prior art methods closest to the present invention are those of Yamashita et al., U.S. Pat. No. 5,384,601 and of Park et al., US Published Patent Application No. 2004/0114798. Both of these documents are incorporated by reference for all purposes as if fully set forth herein.
Both the method of Yamashita et al. and the method of Park et al. operate on color expressed in luminance (henceforth, “luma”) and chromaticity (henceforth, “chroma”) coordinates. Several luma-chroma coordinate systems are known in the art, including the Y-U-V coordinate system, the Y-Pb-Pr coordinate system, the Y-Cb-Cr coordinate system, the L-u-v coordinate system and the L-a-b coordinate system. Yamashita et al. work in the L-u-v coordinate system. Park et al. work in the Y-Cb-Cr coordinate system.
Yamashita et al. define a reference point, in the plane (the “chroma coordinate plane”) defined by their two chroma coordinates, as a reference chroma value that represents typical chroma coordinates of skin, and a rectangle in the chroma coordinate plane that encloses the reference point and that spans the portion of the chroma coordinate plane in which the chroma values of image pixels that depict skin are expected to lie. As shown in their FIG. 8, they adjust chroma values of image pixels by moving the corresponding points (“chroma points”) in the chroma coordinate plane closer to the reference point. A point in the chroma coordinate plane that coincides with the reference point does not need to be moved at all. Similarly, points that fall on the edges of the rectangle and that fall outside the rectangle are not moved. The extent to which a point inside the rectangle that does not coincide with the reference point is move towards the reference point is a smooth function, of the distances between the point and the nearest edges of the rectangle and between the point and the reference point, that vanishes (i.e., equals zero) for points that fall on the edges of the rectangle or that coincide with the reference point.
Park et al. teach a more accurate method of adjusting chroma values that takes into account both the fact that the region of the chroma coordinate plane within which fall chroma values of image pixels that depict skin is better approximated by a tilted ellipse than by a rectangle and the fact that chroma values that are sufficiently close to the reference value need not be adjusted at all. Park et al. define an outer ellipse in the chroma coordinate plane that spans the region of the plane that is populated by chroma values of pixels that depict skin and an inner ellipse that encloses the chroma values that need not be adjusted. Chroma values of image pixels whose chroma coordinates fall in between the two ellipses are adjusted by moving the corresponding points closer to the inner ellipse. The extent to which a point between the two ellipse is moved is a smooth function, of the distances between that point and the two ellipses, that vanishes on the inner ellipse.